1. Field of the Description
The present description relates to data storage and magnetic tape data storage and, in particular, to methods and systems for improving data migration such as from one data storage device (e.g., a first magnetic tape cartridge using a first technology) to a second data storage device (e.g., a second magnet tape cartridge using a second (typically, newer) technology), from a tape library to storage devices accessible over the Internet (e.g., cloud-based storage), or, more generally, from any data storage device to any other data storage device.
2. Relevant Background
For decades, magnetic tape data storage has offered cost and storage density advantages over many other data storage technologies including disk storage. A typical medium to large-sized data center will deploy both tape and disk storage to complement each other and with the tape storage often used for backup and archival data storage. Due to the increased need for securely storing data for long periods of time and due to the low cost of tape, it is likely that tape-based data storage will continue to be utilized and its use will only expand for the foreseeable future. Briefly, magnetic tape data storage uses digital recording on to magnetic tape to store digital information, and the tape is packaged in cartridges and cassettes (i.e., the storage media or simply “media”). The device that performs writing and reading of data is a tape drive, and tape drives are often installed within drive slots of robotic tape libraries, and these libraries may be quite large and hold thousands of cartridges and many drives to provide a tremendous amount of data storage (e.g., each tape may hold several terabytes of uncompressed data).
One difficult issue facing data managers and designers of data storage networks or systems is how to provide data migration in an effective manner. Data migration is the process of making an exact copy of a set of data (e.g., an organization's archived data) from one device to another device and then redirecting all input/output (I/O) activity to the new device. Preferably, the data migration will be planned so active applications are not disrupted or disabled during data migration, and a host application (e.g., a copy or data migration manager software module or application) may be run on a server to manage the process and provide automated migration by transferring data between storage types, formats, and/or computer systems. This host application may act to map data from the old or existing data storage system and its devices (e.g., tape cartridges) to the new data storage system and its devices, thereby providing a design for data extraction and data loading.
There is a wide variety of reasons that an organization may perform a data migration including: (a) storage technology replacement or upgrade (e.g., changeover to new magnetic tape technology); (b) storage consolidation; (c) relocation of a data center; and (d) storage equipment maintenance. Data migrations are performed regularly for many organizations such as quarterly or more often, but even such a routine process such as data migration is not without problems. The problems may include: (a) extended and/or unexpected downtime during data migration (e.g., data migration can be a very time consuming process depending upon the amount of data being migrated and/or number of data storage devices affected by the migration); (b) data corruption or loss; (c) application performance issues during the migration (e.g., data migration may be competing with data processing bandwidth on a server(s)) and after the migration is completed; and (d) compatibility issues after the data migration.
In many cases, data migration presently involves a device-to-device copy (e.g., tape cartridge-to-tape cartridge copy) using a traditional server. The server is communicatively linked to both the source devices and the target or destination devices, and a host application (e.g., a “migration manager”) is run by one or more processors on the server to manage data migration including automating all or portions of the data extraction and data loading processes. FIG. 1 illustrates a conventional data storage system or network 100 during data migration. In this example, data is being copied from a data storage source (e.g., a tape library implemented with a first tape technology) to a data storage target (e.g., a tape library or libraries implementing the same tape technology or, as is common, implementing a newer tape technology).
A communication link device (e.g., a Fibre Channel (FC) switch or other device(s)) 130 is used to provide a digital communication link, as shown with lines 112, 122, between the source 110 and target 120 for this data migration. A server 150 is also linked, as shown with line 152, to data storage source 110 and the data storage target 120 via the link device 130. The dedicated server 150 includes a host adapter (e.g., a host bus adapter (HBA)) that connects the server 150 to the network device 130 and storage devices 110, 120. The server 150 also includes one or more processors 156 that manage the host adapter 154, execute or run software including the host application (e.g., copy or data migration manager 158), and manage data storage devices or memory 160. As shown in FIG. 1, the data path 160 for the data being migrated or copied is from the data storage source (e.g., a first tape cartridge in a first tape library during extraction) 110, through the communication link device 130, to the dedicated server 150, and then back through the communication link device 130 to the data storage target 120 (e.g., a second tape cartridge in a second tape library during loading). The control path 159 is between the host application 158 and other components within the server 150.
A number of problems can arise during data migration in traditional systems such as the data storage system 100 shown in FIG. 1. When servers are used to copy from a source device to a target or destination device, the data is not fully protected, with regard to security and data integrity issues, throughout the entire data path. As shown in the system 100 of FIG. 1, the data path 160 passes from the communication link device 130 to and into the dedicated server 150. Once the data has passed the host bus adapter (HBA) 154 and is being moved within the server 150, there is exposure in memory (or disk) 160 as shown with the migrating data 162 that is in an unsecure (e.g., unencrypted) or potentially corrupt (e.g., missing data) state. Another issue with the use of a server for data migration is that the migration processes being performed by the host application often will have to compete for server or processing bandwidth with other applications/processes running on the server. For example, the maximum available bandwidth may be several hundred megabytes (MB) per second, but this will be reduced if data migration is performed concurrently with other server processes.
Hence, there remains a need for improved methods and systems for performing data migration in a data storage system. Preferably, such methods and systems would be compatible with existing data storage devices (e.g., with tape storage technologies and devices) and with existing host applications (e.g., existing copy or data migration managers) while improving migration efficiency and/or providing increased data security during the entire migration process.